Scalable Inference in Models with Multiple Timescales

In many settings we may wish to learn dynamics at multiple timescales. For example, in the context of speech analysis, we may wish to model both the dynamics within individual phonemes as well as the dynamics across phonemes [68, 18]. In the context of modeling behavior, motion [51], or handwriting [67], it is natural to decompose movements into steps, while still modeling the statistics of the sequence of movements. Each of these modeling tasks involves dynamics at multiple timescales, and therefore it is natural to consider dynamical models that can capture such dynamics while maintaining tractable inference. In this chapter, we develop a Bayesian nonparametric model and associated inference algorithms applicable to unsupervised learning of such dynamics. We combine and build on ideas developed in previous chapters. In particular, we extend the HDP-HSMM developed in Chapter 3 to include Markovian dynamics within each of its segments. The explicit duration modeling provided by the HDP-HSMM allows us to set duration priors that can disambiguate short-timescale dynamics from long-timescale dynamics and is important for identifiability in the unsupervised setting. Using ideas from Chapters 3 and 4, we develop e cient Gibbs sampling algorithms for our proposed model. Finally, extending ideas from Chapter 5, we also develop a structured Stochastic Variational Inference (SVI) algorithm, which allows inference to scale to large datasets. Developing scalable inference with e cient updates is particularly relevant when fitting rich models, since more data are often required to fit more complex models e↵ectively. The main contributions of this chapter are algorithmic, particularly in incorporating the algorithmic techniques developed in previous chapters. While the model we propose is new, as we discuss in Section 6.2 many similar models have been explored in the literature. The key advantage to our model is its amenability to the e cient inference algorithms we develop. Our model also benefits from explicit duration modeling and a